Software on mobile devices is used by billions of users around the world and is frequently updated over limited bandwidth connections, such as cellular data plans. In many cases, software developers may attempt to reduce the size of software updates in order to minimize the amount of bandwidth consumed by such updates. One technique that's frequently applied to software updates is the delta update. In a delta update, rather than sending the entire software package again, the old software is compared to the new version and a delta file describing the changes is produced. The delta update is sent to the user's mobile device where the delta update is applied to the old version of the software to generate the new version of the software. The delta update is typically much smaller than the whole program, and thus bandwidth and storage are saved.
Unfortunately, this approach may not be nearly as effective for neural networks, which are often represented as a group of large numeric matrices. In many cases, each new training or update of the neural network may produce a completely different group of matrices with little relation in values to those trained before, meaning that the delta update approach may not produce significantly smaller files than sending the entirety of the new version of the neural network. The instant disclosure, therefore, identifies and addresses a need for systems and methods for efficiently updating neural networks.